1 

Estimation in Slow Mixing, Long Memory 

Channels 

Meysam Asadi, Ramezan Paravi Torghabeh and Narayana P. Santhanam 



Abstract 

We consider estimation of finite alphabet channels with memory where the transition probabilities 
(channel parameters) from the input to output are determined by prior outputs (state of the channel). 
While the channel is unknown, we observe the joint input/output process of the channel — we have n i.i.d 
input symbols and their corresponding outputs. Motivated by applications related to the backplane channel, 
we want to estimate the channel parameters as well as the stationary probabilities for each state. 

Two distinct problems that complicate estimation in this setting are (i) long memory, and (ii) slow 
mixing which could happen even with only one bit of memory. In this setting, any consistent estimator 
can only converge pointwise over the model class. Namely, given any estimator and any sample size n, 
the underlying model could be such that the estimator performs poorly on a sample of size n with high 
probability. But can we look at a length-n sample and identify ;/ an estimate is likely to be accurate? 

Since the memory is unknown a-priori, a natural approach, known to be consistent, is to use a potentially 
coarser model with memory k n = a„logn, where a n is a function that grows 0(1). While effective 
asymptotically, the situation is quite different when we want answers with a length-n sample, rather than 
just consistency. Combining results on universal compression and Aldous' coupling arguments, we obtain 
sufficient conditions (even for slow mixing models) to identify when naive (i) estimates of the channel 
parameters and (ii) estimates related to the stationary probabilities of the channel states are accurate, and 
bound their deviations from true values. 

Index Terms 

Context-tree weighting, Coupling, Markov processes, Pointwise consistency, Universal compression. 

I. Introduction 

Science in general, information theory and learning in particular, have long concerned themselves with 
parsimonious descriptions of data. One of the driving motivations is that if we understand the statistical 
model underlying the data, we should be able to describe the data using the fewest possible bits — a concept 
that is formalized by the notion of entropy and that forms the foundation for quantifying information. 

A more interesting situation arises when the underlying statistical model is unknown. Depending on the 
application at hand, we then consider a set V of all possible models that are consistent with what is known 
about the application — say all Ltd or Markov models. It is often possible that we can still describe the 
data, without knowing the underlying model other than that it belongs to V, using almost as succinct a 
description as if we knew the underlying statistics. This notion is formalized as universal compression (H, 
0. Of course, the universal description always incurs a penalty — the redundancy or excess bits beyond 
the minimum needed if we knew the model. 

Closely related to universal compression is the concept of Minimum Description Length (MDL), where 
the redundancy above is interpreted as the complexity of the model class. A natural question then is — if the 
redundancy is small (i.e., can be upper bounded uniformly for all models in V by a function sublinear in 
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the data length), in the process of obtaining a universal description did we somehow estimate the unknown 
model in VI 

While the general answer is no, universal compression and estimation often do go hand in hand, at 
least in several elementary settings. Suppose V is the collection of all distributions over a finite set A, 
and the data at hand is a length-n sequence in A n (the set of all length-n strings of symbols from A) 
obtained by independent and identically distributed (i.i.d) sampling from an unknown distribution in V. 
It is then possible to estimate the underlying distribution using a universal description, as is the case 
with the universal estimators by [[3] or (4J, 0). As a more complex example, the Good Turing estimator 
(see 0) can also be interpreted as being obtained from such a universal description [7] in a more general 
setting — data is exchangeable, rather than i.i.d. 

Even with Markov processes, the connections between universal compression and estimation become 
more subtle. At the gross level, compression guarantees hold uniformly over Markov model classes but 
usually not for estimation. Despite this apparent dichotomy, our estimation results rely fundamentally on 
universal compression being possible. Consider a length-n sample obtained from an unknown source in 
the class of binary Markov sources with memory one. No matter what the source is or how the sample 
looks like, there are universal algorithms which describe the sample using at most O(logn) bits more than 
the best description if the source were known. Yet, as we will see, irrespective of how large n is, we may 
not be in a position to reliably provide estimates of the stationary probabilities of Os and Is. 

Let the transition probability from 1 to in our memory- 1 source be e ^ 1/n. By changing the transition 
probability from to 1 appropriately, we can vary the stationary probabilities of Is and Os in a wide range, 
without changing how a length-n sample will look like. 

Example [3] in Section ?? gives two such binary, one-bit memory Markov sources with stationary 
probabilities (1/2,1/2) and (2/3,1/3) respectively. But, if we start from 1, both sources will, with high 
probability, yield a sequence of n Is. We cannot distinguish between these two sources with a sample 
of this size — and hence it is futile to estimate stationary probabilities from this sample. This particular 
phenomenon, where the number of times each state (0 and 1 here) appears is very different from their 
stationary probabilities is often formalized as slow mixing, see HI. 

In this paper, we deal with estimation and operation of channels where, both the input and output are 
from a finite alphabet. The channel state depends on prior outputs alone and there is no feedback. We are 
motivated by the backplane channel, which we will describe shortly. We ask — given n inputs and their 
corresponding outputs, can we estimate the transition probabilities from the input to output, as well as the 
stationary probabilities of various states (or sets of states)? 

We emphasize at the outset that we do not exclude slow mixing of the channel evolution. Instead, our 
philosophy will be: given n samples, what is the best answer we can give, if anything? As the above 
example shows, we have an estimation problem where any estimator can only converge pointwise to the 
true values, rather than uniformly over the model class. One way to get around this impasse is to add 
restrictions on the model space as is done in most prior work. However, very few such restrictions are 
justified in our application. So we take a different approach: can look at some characteristics of our length-n 
sample and say if any estimates are doing well? 

Say, for the sake of a concrete example, that we have a sample xi from a binary Markov process with 
memory 1, with n — logn Is followed by a string of logn Os — so perhaps, this may have come from a 
source as in Example [3] As we saw, it is futile to estimate stationary probabilities in this case. Contrast 
this sample with a new sample X2, also with n — logn Is and logn Os, but X2 has Os spread uniformly 
in the sequence. Unlike in the case of xi, upon seeing X2 we may want to conclude that we have an i.i.d 
source with a high probability for 1. 

The particular application we are motivated by arises in high speed chip-to-chip communications, and 
is commonly called the backplane channel 0. Here, residual reflections between inter-chip connects form 
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a significant source of interference. Because of parasitic capacitances, the channel is highly non-linear as 
well, and consequently the residual signal that determines the channel state is not a linear function of past 
inputs as in typical interference channels. We therefore consider a channel model where the output is not 
necessarily a linear function of the input, and in addition, the channel encountered by any input symbol is 
determined by the prior outputs. Such a model also yields to analytical transparency. Therefore, we begin 
with estimation problems in channels whose state is determined by the output memory. See also JTOl . iTTTTl 
for other examples of output memory channels. 

Our main results are summarized in Section III These results show how to look at a data sample and 
identify states of the channel that are amenable to accurate estimation from the sample. They also allow 
us to sometimes (depending on how the data looks) conclude that certain naive estimators of stationary 
probabilities or channel transition probabilities happen to be accurate, even if the channel evolution is slow 
mixing. To obtain these results, we combine universal compression results of the context tree weighting 
algorithm with coupling arguments by Aldous [12J. 



A. Prior Work on Estimation of Markov Processes 

Estimation for Markov processes has been extensively studied and falls into three major categories (i) 
consistency of estimators e.g., |[T3l . 11141 . |fl5l . iPToll . (ii) guarantees on estimates that hold eventually almost 
surely e.g., ifTTll . |[T8l . and (iii) guarantees that hold for all sample sizes but which depend on the model 
parameters e.g., |[T9ll , 11201 . ETTl . ll22l . As mentioned earlier, performance of any estimator cannot not be 
bounded uniformly over all Markov models, something reflected in the line (iii) of research and in our 
work. The crucial distinction is that we can gauge from the observed sample if our estimator is doing well, 
something that does not hold in (iii). The list is not exhaustive, rather work closest to the approaches we 
take. 

In |[T9ll , EDI . ll22l . exponential upper bounds on probability of incorrect estimation of (i) conditional 
and stationary probabilities and (ii) the underlying context tree, are provided for variants of Rissanen's 
algorithm context and penalized maximum likelihood estimator. However, the introduced deviation bounds 
depend on the parameters {e.g., depth of the tree, stationary probabilities) of underlying process which 
are unknown a-priori in practice. In 11211 . the problem of estimating a stationary ergodic processes by 
finite memory Markov processes based on an n-length sample of the process is addressed. A measure 
of distance between the true process and its estimation is introduced and a convergence rate with respect 
to that measure is provided. However, the deviation bound holds only when the infimum of conditional 
probabilities of symbols given the pasts are bounded away from zero. 

In this work, given a realization of a Markov process, we consider a coarser model and provide deviation 
bounds for sequences which have occurred frequently enough in the sample. In contrast to prior literature, 
while we make an assumption justified by the physical model — that dependencies die down in the Markov 
model class we consider — our bounds can be calculated using only parameters which are well-approximated 
from data. In particular, we do not assume neither a-prior knowledge on the depth of context tree of the 
process nor the conditional probabilities given the pasts are bounded uniformly away from zero. 



B. Prior Work on Information rates of channel with memory 

The computation of the capacity of channels with memory has long been an open problem. The past 
efforts mainly focus on the class of finite state channels. To summarize, researchers have considered (i) 
computing information rate, (ii) finding lower and upper bounds for the information rates, and (iii) capacity 
achieving distributions. A comprehensive review is available in |23h ll24l . ll25ll . ll26l . In particular, for ISI 
channels with an average power constraint and Gilbert-Elliot-type channels, the capacity is already known. 
In addition, the capacity for output memory channels with an additive noise channel (independent of input) 
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was computed in ifTOl . Note that, in this line of work, the channel model and its properties was assumed 
to be known. 

II. Markov Processes and Channels 

A. Alphabet and strings 

Most notation here is standard, we include them for completeness. A is a finite alphabet with cardinality 
|„4|, A* = (Jfc>o-4 fc anc ^ denotes the set of all semi-infinite strings of symbols in A. 

We denote the length of a string u = u\,...,ui £ A 1 by |u|, and use = (uj, • • • , Uj) . The 
concatenation of strings w and v is denoted by wv. A string v is a suffix of u, denoted by v < u, if 
there exists a string w such that u = wv. A set T of strings is suffix-free if no string of T is a suffix of 
any other string in T. 

B. Trees 

As in 1271 for example, we use full A— ary trees to represent the states of a Markov process. We denote 
complete trees T as a suffix-free set T C A* of strings (the leaves) whose lengths satisfy Kraft's lemma 
with equality. The depth of the tree T is defined as d{T) = max{ |u| : u £ T}. A string v E .4* is an 
internal node of T if either v £ Tor there exists u 6 T such that v -< u. The children of an internal 
node v in 7~, are those strings (if any) av, a E A which are themselves either internal nodes or leaves in 
T. 

For any internal node w of a tree 7~, let Tw = {u E T : w ■< u} be the subtree rooted at w. Given 
two trees 71 and 7i, we say that 71 is included in 7i (71 ^ 72), if all the leaves in 71 are either leaves 
or internal nodes of 7i- 

C. Models 

Let 7 ,+ (.4) be the set of all probability distributions on A such that every probability is strictly positive. 

Definition 1: A context tree model is a finite full tree T C A* with a collection of probability 
distributions q s £ T 3 " 1 " (.4) assigned to each s £ T. We will refer to the elements of 7" as states (or contexts 
), and (/(T) = {(7(a|s) : s G 7", a £ .4.} as the set of state transition probabilities or the process parameters. 

□ 

Every model (7~, q{T)) allows for an irreducible, aperiodic^ and ergodic [28]. Such Markov process has 
a unique stationary distribution \i satisfying 

fiQ = H, (1) 

where Q is the standard transition probability matrix formed using q{T). Let p T be the unique stationary 
Markov process {. . . , Yq, Y\, Y2, . . .} which takes values in A satisfying 

Pr. q (Yl\y°oo) = Qfrls) 

whenever s = CfiY®^), where c-r : A°° — > T is the unique suffix s ■< Y®^ in 7". As a note, when we 
write out actual strings in transition probabilities as in g(0|1000), the state 1000 is the sequence of bits 
as we encounter them when reading the string left to right. If follows the state 1100, the next state is a 
suffix of 11000, and if 1 follows 1100, the next state is a suffix of 11001. 

Observation 1: A useful observation is that any model (T,q(T)) yields the same Markov process as 
a model (V, q{T')) where 7~ < V and for all s' £ V, <?(-|s') = ^(•|c r (s / )). □ 

'irreducible since q s £ V + (A), aperiodic since any state s £ T can be reached in either |s| or |s| + 1 steps. 
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Fig. 1. (a) States and parameters of a Markov process in Example [T] (b) Same Markov process reparameterized to be a complete 
tree of depth 2. We can similarly reparameterize the process on the left with a complete tree of any depth larger than 2. 



Example 1: Let (T,q{T)) be a Model with T = {11,01,0} and g(l|ll) = j,?(l|01) = |,g(l|0) = 
|. Fig. [I] (b) shows the Markov process as a model (T',q(T r )) with T' = {11,01,10,00} satisfying 
conditions in Property [1] □ 

D. Channel Model 

We focus on Markov channels defined as follows. Both input {X n }^ =1 and output {Y n }^ =1 are finite 
alphabet processes taking values in A and the state of channel in each instant depend on sequence of prior 
outputs of the channel. The input process is drawn from an i.i.d process, namely P(X n = a) = p a for all 
n E N and a £ A, provided that ^ a&j ^p a = 1- We assume that there is no feedback in this channel setup. 
The joint probability distribution of the channel is 

n 

P(xi,vi) = tlP(xk)P(yk\yi-£,x k ). 

k=l 

We consider the process {(X n , Y n )}™ =1 , and model it as a Markov process p T . Here, T will be a 
complete |.4|-ray tree. To every state s E T and (a, b) E A x A, let 

£,b) = p{Xk = a ^ Yk = 6|c r (y^- 1 ) = s), Vfc e N 
where C7-(y^~^) ^ T (defined above) is the state of the channel at time k. For convenience, we define 

6 s (b\a) = P(Y 1 = 6|c r (y° 00 ) = s,Xi = a). 

The set S = {9 s (-\a) : a E .4} is called the set of all conditional probabilities associated with state s. As 
a remark, for all a E A and s E T, # s ( - |a) £ V + (A) and corresponds to transition probabilities when 
the state of channel is s and input symbol is a. The set ©7- = {0 S : s E T} is the set of all transition 
probabilities of channel model and is called channel parameters. Therefore, we have 

qi a ' b} = p a s {b\a). 

Since the input is a known i.i.d process, estimating q(T) and ©7- are completely equivalent and can be 
obtained from each other using the equations above. 

As emphasized in the introduction, we do not assume the true channel model is known nor do we assume 
it is fast mixing . We would like to know if we can estimate the channel parameters and the stationary 
probabilities of various states of the channel even when we are in the domain where the mixing has not 
happened. 
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III. Main Results and Remarks 

Motivated by the operation and estimation in backplane channels, we consider the following abstraction. 
We have n i.i.d input symbols and their corresponding outputs, and the input/output pairs evolve as 



described in Section II-D above. There is no feedback and the input is a known i.i.d process. The channel 
input/output pairs then form a Markov process p T where both the set of contexts T, and the transition 



probabilities q(T) (or equivalently the channel parameters ©7-, see Section II-D 1 are unknown. 

Using this sample, we want (i) to approximate as best as possible, the parameter set ©7- (ii) the stationary 
probabilities /i(s) of observing an output string s G T, and (iii) estimate or at least obtain heuristics of 
the information rate of the process. 

Two distinct problems complicate estimation of ©7- and the stationary probabilities. First is the issue 
that the memory may be too long to handle — in fact, if the source has long enough memory it may not be 
possible, with n samples, to distinguish the source even from a memoryless source (Example [2]). Secondly, 
even if the source has only one bit of memory, it may be arbitrarily slow mixing as in the example in the 
introduction and in Example [3] No matter what n is, there will be sources against which our estimates 
perform very poorly. 

A natural way to deal with T being unknown is to try estimating a potentially coarser approximation 
p^ _ to the true model, where T = A k ™ for some known k n . With the benefit of hindsight, we take 
k n = (log re) and write k n = a n logre for some function a n = 0(1). This scaling of k n also reflects the 
well known conditions for consistency of estimation of Markov processes in ||T5l . 

The input/output pairs obtained by the coarser channel model will be a stationary Markov process. 
We call it the aggregation of p T q and is specified in Definition [2] The aggregation matches all joint 
distributions of p T involving variables that are less than k n apart from each other — namely if ii, ... ,ii 
are any increasing sequence of I numbers such that i\ — i\ + 1 < k n , then p T (X^Y^, . . . X^,!^) = 
p T (Xjj,!^, . . . ,Xi n Yi t ). We show that the information rate corresponding to the aggregation p T is a 
lower bound on the information rate of the true channel input/output process p T q in Theorem [3] While 
it may not be possible to directly use this Theorem [3] we develop of the notion of a partial information 
rate, a useful heuristic when the source has not mixed. Moreover, Theorem [3] also motivates the estimation 
questions that form our main results. 

The issue remains that the sample of channel input/outputs (given past K^) at hand is from p T , not 
from p T . To obtain the parameters of the aggregated model, 0^, we use a naive estimator — we simply 
pretend that our sample was in fact from p f . Equivalently, we assume that for any output sequence w G T, 
the subsequence of output symbols in the sample that follow w and associated with the same input letter a 
is i.i.d. For example, if our sample has output 1101010100 for input 1111100000, and w = 10, the output 
subsequence following w is 1110, and these outputs correspond to inputs 1000 respectively. We would 
then simply estimate #io(l|l) = 1 and #io(l|0) = 2/3. If p T were arbitrary, there is no reason that this 
naive approach has to be accurate. 

However, it is reasonable given our physical motivation that the influence of prior outputs dies down 
as we look further into the past. We formalize this notion in ([3]) with a function f(i) that controls how 
symbols i locations apart can influence each other, and require Yli^i /(*) < 00 • With this assumption, we 
obtain G C T, a set of good states or good strings of channel outputs (strings in T may not be states 
of p T , but we abuse notation here for convenience) from the sample, and show that with probability 
> 1 — 2lAl kJ+i losn (conditioned on any past Y^), for all a G A and s G G simultaneously 



, ■x / 1 s ; , , ,11 / In 2 In 2 
|0 B (»-0 s (-|a)||i <2./ i + 



log n log ^ 



where = ^2 i>k f[i) and 9 s (-\a) is the naive estimator of 6 s (-\a) described above. In Theorem |7] we 
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strengthen the convergence rate to polynomial in n if the dependencies die down exponentially. 

The above estimation result associated with s G G relies not on mixing but on the fact that Markov 
sources with memory k n can be universally compressed. Therefore, it is still definitely possible that the 
counts of s G G are nowhere near their stationary probabilities. How do we then interpret their counts? To 
answer this question, we define a parameter r] e in (|7]) that is calculated using just the parameters associated 
with s6G. Suppose {u-i} is summable as well, and let = Y^i>j v i- A coupling argument on a natural 
Doob Martingale construction then proves that if the evolution of p T restricted to just states in G is 
aperiodic, then for all t > 0, Y®^ and w G G the counts of w in the sample, iV n (w), concentrates: 

;M(W) |N+Iv0 ^n^J t2 ( r ?J"( 1 - ' 




p T (\N n (w) - h^-J- I > ilF") < 2exp , 
r,U V 7 KG) \ 2h\U n + r], 

where l n is the smallest integer such that <&^ n < -, n is the total count of good states in the sample and 
fi denotes the stationary distribution of p T q . 

IV. Background 

A. Context Tree Weighting 

Context tree weighting algorithm is a universal data compression algorithm for Markov sources E7L 
||29ll , and the algorithm can be used to capture several insights into how Markov processes behave in the 
non-asymptotic regime. Let y\ be sequence of symbols from A. Let / = A D for some positive integer 
D. For all s G T and a G A, let be the empirical counts of string sa in y™. The depfh-Z) context tree 
weighting constructs a distribution 

p c(y ri y %)>2H^ +ii -«nn(y^ 

Note that no Markov source with memory D could have given a higher probability to than 

nrr 

se f aeA 



So, if |^4| D logn = o(n), then p c underestimates any memory-D Markov probability by only a subexpo- 
nential factor. Therefore, D = O(logn) is going to be the case of particular interest. 

B. Coupling for Markov Processes 

Coupling is an elegant technique that will help us understand how the counts of certain strings in a 
sample behave. Let p T be our Markov source. A coupling u for p T is a joint process {Y n ,Y n } such 
which both {Y n } and {Y n } are marginally faithful evolutions of p T Say {Y n } and {Y n } here are copies 
of p T q that were started with states s, s' G T, respectively. The joint distribution u({Y n , Y n }) is otherwise 
arbitrary, but we want to encourage them to coalesce. For w G T, iV n (w) (iV n (w)) is the number of 
occurrences of w in {Y n } ({Y n }). Then, |E P N n (w) — E p iV n (w)| equals 



J2E 0J [t{cr(Yi oo )=w)-l{c T (Yi o0 )=w)] <J2 



i=i 



i=l 

n 



E^mcriYi^) = w) - i(c T (rj = w)] 



<^ w (c r (rj/c T (fJ). 



i=l 



For tutorials, see e.g., |30], 0, OH . 



February 25, 2013 



DRAFT 



8 



V. Long Memory and Slow Mixing 



There are two distinct difficulties in estimating Markov processes as the ones we are interested in. The 
first is memory that is too long to handle given the size of the sample at hand. The second issue is that 
even though the underlying process might be ergodic, the transition probabilities are so small such that 
the process effectively acts like a non-ergodic process given the sample size available. We illustrate these 
problems in following simple examples. 

Example 2: Let T = A k denote a full tree with depth k and A = {0, 1}. Assume that q(l\0 k ) = 2e 
and q(l\10 k ~ 1 ) = 1 — e with e > 0, and let g(l|s) = \ (where k indicates a string with k consecutive 
zeros) for all other s G T ■ Let p T represent the stationary ergodic Markov process associated with 



this model. Observe that stationary probability of being in state is 



2 k + 1 -l 



while all other states have 



stationary probability 



2 fc+i_i 



Let Yp be a realization of this process with initial state l k < Y-oo- Suppose 



k S> u; (log re). |^] With high probability we will never find a string of k — 1 zeros among n samples, and 
every bit is generated with probability 1/2. Thus with this sample size, with high probability, we cannot 
distinguish p T from an Lid Bernoulli(l/2) process. □ 
We therefore require that dependencies die down by requiring that channel parameters 8f s and 6q s , 
corresponding to sibling contexts Is and 0s, satisfy Q in Section VI 

Example 3: Let A = {0, 1} and T = {0, 1} with g(l|l) = 1 - e, and g(l|0) = e. For e > 0, this 
model represents a stationary ergodic Markov processes with stationary distributions /x(l) = j,/i(0) = \. 
Let T 1 = {0, 1} with g'(l|l) = 1 — e,q'(l\0) = 2e. Similarly, for e > this model represents a stationary 
ergodic Markov processes with stationary distributions //(l) = |,//(0) = |. Suppose we have a length-re 
sample. In this case, we cannot distinguish between these two models if e <C o(l/re). □ 



A. Lower Bound on Information Rate 

Consider a channel with state tree T and parameter set ©7-. Suppose that d(T) = D < 00. The 
information rate for an Ltd input process with P(Xj i = a) = p a for this channel is 

R T = lim -I(X n ;Y n ) 

n— >oo n 

lim ~[H(Y n ) - H(Y n \X n )] 



n— >oo n 



1 n 

lim — y 
n— >oo re ^ — ' 



k=l 
D 



H{Y k \Y k - l )-H{Y k \Y*-\X k ) 



k-l 



lim 1 EE p ( yfe_1 = ^ 1 ) 

n— >oo n L — ' £ — ' 

fc=l y k - 1 

+ iim - y y p(Y k - 1 -- 

n—>-oo n — ' ^ — ' 
k=D+ly k - 1 

hm-yyy p{Y k - 1 



H(Y k \Y k ~ l = - H(Y k \Y k - L = y k - L ,X k ) 



k-l _ „.fc-l 



,k-l\ 



H{Y k \Y k - 1 = y k - x ) - HiY^Y 1 *' 1 = y k ~\X k ) 



y 



H(Y k \Y 



k-l „.fe-l^ 



H(Y k \Y k - 1 =y k -\X k ) 



where (a) is by chain rule for entropy and the fact that input process is independent of output process. 
This condition always is correct as long as there is no feedback in the channel and (b) is straightforward 



2 A function /„ = u(g n ) if lim„^oo /„ 



February 25, 2013 



DRAFT 



9 



from definition of the conditional entropy. The equality in (c) holds since the first term in (b) vanishes as 
n — > oo and observing that for k > D + 1, 

A k - l = |J {y^e^-^s^/- 1 }. 
seT 

Note that for all k 6 N, if s ■< y k ~ 1 , it can easily be shown that 

9 s (b\a) 



HiYklY*- 1 = - H(Y k \Y 



'- 1 =y k -\X k ) = Y J Pa^O s (b\a)log 



aeA beA 



T,a'€APa' e s( b \ a ') 



def 



Therefore, 



R s (e s ). 

Rr= iim l - £ y: e p^*- 1 = ^^ce.) 

fc=D+lseT j / fc - 1 : 

r 1 n 

=£«■<*> E E nr" 



k-U 



seT 



y 



E*(80 E 

seT 



fc-i\ 



]>>(s)i? s (e s ). 



seT 

The equality in (d) is by changing the order of summations and (e) follows by using Cesaro's lemma l32l . 
Finally, (f) holds by properties of stationary distribution in Markov processes. As a remark, note that for 
fixed input distribution, R s is a function of S = {9 s (-\a): a G A}. 

Lemma 1: For fixed input distribution, R s (@ s ) is convex in S . 
Proof Let A e [0, 1] and A = 1 - A. Let G s = {0 s (-\a) : a £ A} and Q' s = {0' s (-\a) : a e A} be two sets 
of valid conditional distributions associated to state s. Then, 



= £*>»£ 

aeA &6A L 

= E^E 

aeA &6A L 

^E^E 

aeA 6eA L 



X9 s (b\a) + X9' 8 (b\a) log 



A# s (6|a) + A# s (&|a) 



A0 s (6|a) + Ar^(&|a) log 



A6» s (6|a) loe 



E ?V A0 s (&|a') + A^(&|a') 

a'eA V 

Afl s (b|a) + A%(&|a) 

A Z Pa'0 s (b\a>) + \ £ p a ^(&|a') 
a'eA a'eA 

- + A6> s (6|a)log 



A- E Pa'^s (6|c 
a'eA 



A- E P.'W)J 
a'eA 



= Ai? s (e s ) + Ai? s (G' s ) 

where the inequality in (e) follows by using log-sum inequality (see e.g., |[32l ). □ 
Since the memory is unknown a-priori, a natural approach, known to be consistent, is to use a potentially 
coarser model with depth k n . Here, k n increases logarithmically with the sample size n, and reflects lfl31 
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I 




Fig. 2. (a) Markov process in Example [4] (b) Aggregated model at depth 1. From Observation 1, the model on the left can be 
reparameterized to be a complete tree at any depth > 2. We can hence ask for its aggregation at any depth. Aggregations of the 
above model on the left at depths > 2 will hence be the model itself. 



well known results on consistent estimation of Markov processes. We show that coarser models formed 
by properly aggregating states of the original channel are useful in lower bounding information rates of 
the true channel. 

Definition 2: We say that p T . aggregates p T q (or p T q refines p- .), if T ^ T and p r be the 
stationary Markov process with state transition probabilities given by 

i x Ev6T w M v M a l v ) 
?(ow) = — ^ —: — 

£veT„ M v ) 

for all w G T and a £ A, where \x is the stationary distribution associated with p T q . Using Observation [TJ 
wolog, no matter what T is, we will assume p T q has states T such that T -<T ■ □ 

Example 4: Let p T be a Markov process with T = {11,01,0} and g(l|ll) = J,g(l|01) = 
|,g(l|0) = |. For this model, we have //(ll) = ^,/i(01) = ^ and /t(0) = Fig. & (b) shows 
an aggregated process p t , with f = {1,0}. Notice that 1) = (^3 + + |p = ^ and 

g(l|0) = |. ""' " " " □ 

Lemma 2: Let p T q be a stationary Markov process with stationary distribution fi. If p r . aggregates 
p T q then it has a unique stationary distribution /i and for every w G T 

A(w) = ^ m(v). 

veTw 

Moreover, for all G .4 fc such that is an internal node of T we have fi{a\) = n{a\). 

Proof Let Q be the transition probability matrix formed by the states of p T . First notice that by definition, 

for all w, w' G T 

Jo if $a G A s.t. w ^ w'a 

1 q(a\w') if w ^ w'a for some a G A 



Q(w\w') 



Since p T q is irreducible and aperiodic, p T _ will also be irreducible and aperiodic and thus, has a unique 



stationary distribution fx. Hence, there exists a unique solution for 

/i(w) = /t(w')(5(w|w') Vw G T ■ (2) 



w'eT 

We will consider a candidate solution of the form 



ver w 
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for every w G T and show that this candidate will satisfy (|2|). Then, the claim will follow by uniqueness 
of the solution. To show this, note that for Vw G T 



E M(w')<zWw') = 



w'eT 

w-<w'a 



w'eT 
w^w'a 



E /*(v) 



ver w 



Ever , M V M°I V ) 



Ever, A*(V) 



E E A i ( v )^( a l v ) 



va 



w'eT ve7 ^ 

- £ Em 

w'eT ve7 ^ 

= E Ms) 

seT w 
= A(w) 

where (d) follows by observing that 

w H {va: ]w'6T,a£i s.t. w ^ w'a and v G T, }, 

and then using properties of the stationary distribution of p T . Note that the second statement of Lemma 
automatically follows from the uniqueness of stationary distributions. □ 
In a similar manner as Definition |2j given any input output process for a channel we can define an 
aggregated channel with tree T and parameter set 0^. For all w G T, let G w = {# w (-|a) : a G .4} in 
which for fixed a £ A, # w (-|a) is given by 

e w (b\a) = — = — — , V6G.4 

Ev'eT w M V ) 

Theorem 3: Consider a channel with tree T and parameter set ©7-. If T aggregates T, then i?^ < R T . 
Proof Note that for all w G T, %, = {s G T : w < s}. Since T ^ 7", we have 



% = E A(w)i?w(e w ) 



weT 



< E ^ w ) E 



M(v) 



£ v 'eT w MV) 



y-iivCOv) 



= E E M(v)i2v(6 v ) 
wer ve7 ^ 

= ^/x(s)i?s(G s ) 
seT 

= J R T 

where the inequality in (a) holds by Lemma [T] and the fact that Va, 6 G A 

M°M = — ^ rr, — • 

£ver„ M v ) 

The equality in (b) hods since ytt(w) = X] v 'eT M v ')- 



□ 
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Remark In this paper, we are particularly concerned with the slow mixing regime. As our results will 
show, in general it is not possible to obtain a simple lower bound on the information rate using the data 
and taking recourse to the Theorem above. Instead, we introduce the partial information rate that can be 
reliably obtained from the data 



where G C T will be a set of good states that we show how to identify. The partial information rate is 
not necessarily a lower bound, but in slow mixing cases it is sometimes the best heuristic possible. We 
systematically handle the information rates of slow mixing channels using the estimation results below in 
different paper. □ 

Notwithstanding the previous remark, we will focus on estimating the aggregated parameters ©7-, where 
T = A n has depth k n where k n grows logarithmically as o ra logn for some a n = 0(1). Now pj- ~ is 
unknown and we do not have access to samples from it. And there is, of course, no guarantee that the 
counts of short strings are any more reliable in a long-memory, slow mixing process. 

Example 5: Let T = {11, 01, 10, 00} with g(l|ll) = e, ?(1|01) = \, g(l|10) = 1 - e, g(l|00) = e. If 
e > 0, then p T is a stationary ergodic binary Markov process. Let \i denote the stationary distribution of 
this process, respectively. A simple computation shows that /x(ll) = jz^- e , /-t(Ol) = fE§§> A^(10) = fE§f 
and M (00) = fE|, and M (l) = ^_ + |=| = 3^ and m = 2^ + V| = ^ 

Suppose we have a length n sample. If e <C i, then /i(l) w = and /i(0) w | respectively. If the initial 
state belongs to {11, 01, 10}, the state 00 will not be visited with high probability in n samples, and it can 
be seen that the counts of 1 or will not be near the stationary probabilities p,(l) or /i(0). For this sample 
size, the process effectively acts like the irreducible, aperiodic Markov chain in Fig. [3 (b) which can be 
shown to be fast mixing. Therefore, the stationary probabilities of the chain in Fig. 



(b), 



Mi)+Moi)' 



^ 10 ^ — and m+Tfrn conver g e quicker than //(l) or p,(0). □ 



m(i)+m(oi) m(1)+m(oi) 



VI. Estimation of Channel Properties 



As noted before in Example [2| if the dependencies could be arbitrary in a channel model, we will not 
estimate the model accurately no matter how large the sample is. Keeping in mind Observation [T] we 
formalize dependencies dying down by means of a function / : Z + — > R + with /(*) < 00 anc ^ 
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assuming that for all w G A* and all c, d G A 

i O cw (b\a) 



, l|</(|w|) (3) 

where a,&Gi and 9 cw (b\a) = P(Y\ = b\c-j-(Y2 OQ ) = cw,Xi = a). Note that the tree is finite iff there 
exist a finite D such that f{i) = for all i > D. 

As mentioned in the last section, we will focus on set of the aggregated parameters at depth k n , Q^k n 
where k n = a n log n. If fe n is large enough, these aggregated parameters start to reflect the underlying 
parameters ©7-. Indeed, by using an elementary argument we will show that both the underlying and 
aggregated parameters will then be close to the empirically observed values for states that occur frequently 
enough. 

Throughout this Section, we assume that we start with some past Y®^, and we see n samples (Xf, YJ 1 ) 
from the channel. All confidence probabilities are conditional probabilities on YJ 1 given Xf and Y®^, but 
we do not write Y®^ out explicitly to avoid cluttering notation. The results hold for all Y®^ (not just 
with probability 1). 

Lemma 4: Let {/(i)}^ be a sequence of real numbers such that there exists some no G N for which, 
< f(i) < 1 for all i > uq. Then, Vj > no, we have 

i-E/ Ws na-/»)^n5(iT7w) 

Proof Let Ei be a sequence of independent events such that F(Ei) = f(i). Then, Vj G N such that 
j > no, we have 

1 - n c 1 - / w) = nU ^) ^ E p (^) = E / w- 

i>j £>j i>j £>}' 

Hence, 

i-j2f(i)<U(i-m). 

i>3 i>3 

Since by assumption < f(i) < 1 for all i > no, the second inequality can easily be derived by the fact 
that 

n(l-/ 2 «)<l. 

i>j 

Therefore, 

□ 

Lemma 5: Let T be a model associated with the channel and satisfying condition Q. Suppose T -< J~ 
with d(T) = fen. If Ei>fe„ /(*) < 1. then for all w G T and a, 6 G ^4 

1 - £ /«) maxtf s (6|a) < 9 w (b\a) < mhl ^0 s (b\a 



Proof Let w G T and fix a, 6 G A Note that for i = fe n , by assumption we have for all c,d £ A 

cvj(b\a) 



1 

/ c ' W (6|o) 



< /(fen)- (4) 
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According Lemma|2j 6 w (b\a) is weighted average of 9 cw {b\a), c G A. Hence, there are largest 6dw{b\o) 
and smallest ^ w (6|o) such that 

e dw (b\a) < § w (b\a) < 9 d , w (b\a). (5) 
Combining (|4]) with (|5]) and straightforward elementary algebra shows that \/c £ A 

e w (b\a)(l - f(k n )) < 9 cw (b\a) < (1 + f(k n ))e w (b\a). 
Proceeding inductively, for all s G 7^ we have 

H {i-m)\e w (b\a)<e a (b\a)< (]J (i + /(0))m%)- 

Now, first part of the lemma implies that 

(l - ^ m) ™*0s(b\a) < 9 w (b\a) < / mins£7 "^ s(b|a 

□ 

Definition 3: For all sequences (X™, Y") obtained from the channel model p , let 7" = -4 fc " with 
&n = a n logn for some function a n = 0(1). For s G T, let Yg be the sequence of output symbols that 
follows the output string s, and correspond to the input x = a. Hence, the length of Y^ is 

n 

N n (s, a)=J2 HM Y -™) = s > X * = fl }> 

k=l 

the number of occurrences of symbol b in Y" is n s (b,a), where 

n 

n s (b, a) = J2 Hct( Y -™) = s ' Y k = b > X k = a}. 

k=l 

We define the naive estimate of 9 s (b\a) as 

9 ^ b a ) = AT ( a \ 

N n {s,a) 

Furthermore, let N n (s) = J2aeA -^«( s ' a )- 1=1 
Remark Note that Yg is i.i.d only if s G T, the set of states for the true model. In general, since we 
do not necessarily know if any of n s (b,a) are close to their stationary frequencies, there is no obvious 
reason why 9 s (b\a) shall reflect 9 s (b\a). □ 

Let Uj = Y^ti>j /(*)• Note that Vj — > as j — > oo and that — Uj \ogVj — > as Vj — > 0. 
Definition 4: Given a sample sequence with size n obtained from the channel model p T , we define 
the set of good states, denoted by G, as 

G = {w G T : Va G A, N n (w,a) > max {nu kn log — , log 2 n} }. 

Remark Note that a state is good if the count of the state is > n a " log 2 n = 2 k ™ log 2 n. Therefore, 
if 2 fc - log 2 n > n, or equivalently k n > log n — 2 log log n, no state will be good and the Theorem below 
becomes vacuously true. This is not a fundamental weakness in this line of argument — it is known that 
k ri has to scale logarithmically with n for proper estimation to hold. □ 
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Remark Observe also that because we do not assume the source has mixed, the theorem below does 
not imply that the parameters are accurate for contexts shorter than k n . This is perhaps counterintuitive at 
first glance, but the below result holds not because of mixing, but because of the fall-off of dependencies. □ 

Theorem 6: Let k n = a n logn. With probability (conditioned on Y®^) > 1 — 2lA|fc „+i losn > f° r a ^ 
a € A, Y®^ and s € G simultaneously 



,,^/,% j ,, mi / In 2 In 2 
0s > " l < 2 J: + . r 

V gn g i 

Proof As before, let = Yli>k /CO an( ^ l et n ^ e iar g e enough that Vk n < \. Note that Lemma [5] 
implies that for all sequences (2:1,2/1) G .A™ x „4 n (all steps hold for all Y®^) 

Pr,M\^Y-oo)< )n U II e s (b\aT^ 

< JJ Yl 9 s (b\a) n ^ 

s <=fa,b£A 

1 \n ^ Ant 



where O s (b\a), n s (b,a) are defined in Definition 3| and the second inequality is because (tz^)™ < 4 

n s (b,a) 



whenever < t < |. Now, let i? n be the set of all sequences that satisfy 



4- n n a<i<o- m £ n,et ^^:' 

Now using a construction similar to the context tree weighting algorithm ll27l . we obtain a distribution 
p c satisfying 

Hence, for all sequences in B n , we have 



> 



2|^| fc ™ + i logn 
4 (n, fe „ + |^|- + Mogn) n set n a ,6^^(6|a)^^) 



2l-4| fc " + 1 logn 

>p r . g (^l^,y° 00 )2i' 4 i fc " +110 ^. 

Thus, £? n is the set of sequences sucn that p c assigns a much higher probability than p T . Such a set 
B n can not have high probability under p T . 



p 7 



(B n ) =p T \y n 1 -.pMx n ll Y» 00 ) >p r J^I^,K 00 )2l- 4 l fc " +110 ^ 



< ^ p c (^|x?,y oo )2-^ fc " +11 °g"<2-i- 4 i fc " +llo g". 



Therefore, with probability > 1 — 2 lo s n , we have 

11 11 P s(0|O; ^ A(\A\^ + n OR n+nu k ) 



4(|>l|*n+i i ogn+ni/fcn ) 
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which implies simultaneously for all s G T and for all a G A 

- A(\A\"n+ilogn+nu kn )- 

beA 

The above equation implies that 9 s (-\a) and 9 s (-\a) are close distributions, since we can rearrange (take 
logarithm and divide both sides by N n (s,a)) to obtain 

\- Mb, a) , n ^n M|5 M A .2(|^»+ 1 logn + Wfe J 

2^ aTTT^ lo S 27^T = D[e s (-\a)\\e s (.\a) < 



beA 



N n (s,a) b 6 s (b\a) \ / j iV„(s,o) 



where the first equality follows by writing out the value of the naive estimate, 6 s (b\a) = n s (b, a)/N n (s, a). 
Since (see for example 11321 ) 

' %^a)-e a ^a)\\l<D(§ B {-\a)\\e a (-\a 



2 In 2' 

for all s G T and a G A, we now have with confidence bigger than 1 — 2~l- 4 l fc " +1 lo s n that 



,(-|a)-0 s (-|a)||i < 



(ln2)(\A\ k »+ 1 logn + nu kn ) 



N n (s,a) 

The Theorem follows from our Definition [4] of good states. □ 
When the dependencies among strings die down exponentially, we can strengthen Theorem [6] to get 
convergence rate polynomial in n. 

Theorem 7: Suppose f[i) = 7* for some < 7 < 1. Let Q be a nonnegative constant such that 
C > 1 ■ For k n = , ' og ? and define 

* — log A— log 7 " log A— log 7 

G = {weT:VaeA iVn(w,a) > n c+I5 ^^}. 

~ I . I fc n +1 . 

Then, for all s E G with probability greater than 1 — 2~I- 4 I iogn simultaneously 



, w -, n £ M mi ln2-((l-7)|^|logn+l) 

||0.(.|a) - 0.{.\a)\\i < 2^ ^—^ . □ 

Proof The proof is similar to Theorem [6| but involves more careful but elementary algebra specific to 
the exponential decay case. 

Remark According to definition of good states in Theorem [7] and the fact that d(7~) = k n , we obtain 

~ log T ~ log 1^1 

\G\ <n log i-ai-i°8t , |7~| = n log i^i-i°st 

implying that if 7 < 1/|»4|, all states of T can potentially be good. □ 
Note that the parameters 9 associated with any good state can be well estimated from the sample while 
the rest may not be accurate. From Example [3] we know that the stationary probabilities may be a very 
sensitive function of the parameters associated with states. It is therefore perfectly possible that we estimate 
the parameters at all states reasonably well, but are unable to gauge what the stationary probabilities of 
any state may be. How do we tell, therefore, if we can trust our naive counts of states? 
To find deviation bounds for stationary distribution of good states, we construct a new process {Z m }^ =l , 
Z m G T from the process {Yn}^^ If Y Um is the (m + l) th symbol in the sequence {Yn}^ =1 such that 
VfO^oo) e men 2"m = c t(^-S>)- The strong Markov property allows us to characterize {Z m }^ =1 as 
a Markov process with transitions that are lower bounded by those transitions of the process {Yn}^^ that 
can be well estimated by the Theorems above. More specifically, let To = min{j > : c^Yi^) G G} 
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and let Zq = cq-{Y-^)- F° r a U m > 1, T m is the (m + l)'th occurrence of a good state in the sequence 
{Y n }n=i> namely 

T m = mm{j > T m _! : CffYi^) G G}, 

and Z m = C7-(y5™ D ). Note that T m is a stopping time [HI, and therefore {Z m }^ =1 is a Markov chain 
by itself. Let B = {s G T : s ^ G}. The transitions between states s, s' G G are then the minimal, 
non-negative solution of the following set of equations in {Q(s|s") : s" G _4. fc ",s G G} 

Q(s|s') =p Tq (c f (Yl 00 ) = s|c t (y° 00 ) =s') + E P^M^-oo) = 8"l«f C^oo) = "WlO 

s"eB 

An important point to note here is that if s and s' are good states, 

Q(s|s') > p T , q {c f ( Y -oo) = sMl^) = s'), 

and the lower bound above can be well estimated from the sample as shown in Theorem [6] 

Property 1: A few properties about {Z m } are in order. {Z m } is constructed from an irreducible 
process {Y n }, thus {Z m } is irreducible as well. Since {Y n } is positive recurrent, so is {Z m }. But despite 
{Y n } being aperiodic, {Z n } could be periodic as in the Example below. But periodicity of {Z m } can be 
determined by G alone (because T, while unknown, is a full, finite ,4-ary tree). □ 
Example 6: Let {Y n } be a process generated by context tree model p T with T = {11, 01, 10, 00} and 
g(l|ll) = i , g(l|01) = e, g(l|10) = 1 — e, g(l |00) = \. If e > 0, then p T represents a stationary aperiodic 
Markov process. If {Z n } be the restriction of process {Y n } to G = {01, 10}, the restricted process will 
be periodic with period 2. □ 
Property 2: Suppose {Z m } is aperiodic. Let [iy and p,z denote the stationary distribution of the 
processes {Y n } and {Z rn }, respectively, withn samples of a sequence {Y n } yielding m n samples of {Z m }. 
Similarly, let /i^(s)denote the stationary probability of the event s < Z. Then for all s,s' G G with 



/ \ i- sr^n l(c r (y_! oc )=s) ^ m l(s-<Z;) , \ 

PY{S) wpl linWoo 2^i=l n _ llm m„^oo Li=l m W P} M s ) 



□ 



For any (good) state s, let G s C .4 be the set of letters that take s to another good state, 

G s = {b G A : c f (sb) G G}. (6) 

Our confidence in the empirical counts of good states matching their (aggregated) stationary probabilities 
follows from a coupling argument, and depends on the following parameter 

r\ e = min min {g(6|u), q(b\v)}. (7) 

u ' ve( ^6eG u nG v 

Note that for the channel model p T , we have 

9(b\u) = £ P(Xi = a)P(Y 1 = b\c f (Y^) = u ,X 1 = a) = J><A( & l a )- 
aeA aeA 
Again, Theorems [6] and [7] allow us to estimate rj 6 from the sample since it only depends on parameters 
associated with good states. The counts of various w G G now concentrates as shown in the Theorem 
below, and how good the concentration is can be estimated as a function of i] 6 (and u^) and the total 
count of all states in G as below. Now G as well as rj e are well estimated from the sample — thus we 
can look at the data to interpret the empirical counts of various substrings of the data. Let $j = >~2i>j u i- 
For the following theorem, we require v\ to be summable. Thus, &j is finite for all j and decreases to 
as j increases. If /(£) ~ j 1 , then &j also diminishes as 7 J . But f(i) ~ i diminishes polynomially, then 
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diminishes as l/j r ~ 2 . If f(i) = l/i 2+v for any i] > 0, we therefore satisfy the summability of V{. 
However, f(i) can also diminish as l/(i 2 poly (log i)) for appropriate polynomials of logi for the counts 
of good states to converge. 

Theorem 8: If {Z m }^ =1 is aperiodic, then for any t > 0, Y®^ and w £ G we have 



2 N 



p r HiV n ( w ) _ | > tjy^) < 2exp 



fen, 



KG) 1 I 2n\4£ n + 7#.(l-$ fe J 

where ^ n is the smallest integer such that <&^ n < ^ n is the total count of good states in the sample and 
\i is the stationary distribution of p T . 
Proof We define 

V i = E[N n (w)\Z Z 1 ,...,Z i ], 

and observe that -jVj}™ =1 is a Doob Martingale. Note that V = E[N n (w)\Z ] and Vn = N n (w). 
Remark To summarize, we first bound the differences \Vi+i — Vi\ of the martingale using a coupling 
argument on two copies of the chain {Z n }. Since the memory of the process p T could be large, in our 
proof the coupled chains never actually coalesce in the usual sense but enough that the chance they diverge 
again within n samples is less than l/n. This is where the parameter i n comes in as well. Once we bound 
the differences in the martingale {Vi}™ =1 , the theorem follows as an easy application of Azuma's inequality. 

□ 

Now since for alH > 

\Vi - Vi-i\ = |E[iV n (w)|Z , . . . M ~ E[N n (w)\Z , . . . ,Zi- X ]\ 



< max 

z',z" 



E[JV n (w)|Z 0> . . • ,Z'i\ ~ E[7V n (w)|Z , . . . X\ 



we bound the maximum change in N n (w) if the i'th good state was changed into another (good) state. 
To do so, we use a coupling argument as follows. Let G be the set of good states from Definition |4j and 
suppose good states occur n times in the sequence. Suppose there are sequences {Z, } (starting from state 
Z { ) and {Z[ '} (starting from state Z[), both faithful copies of \Zj \ yet coupled with a joint distribution uj 



to be described below. From the coupling argument of Section IV-B we have for w € G (hence |w| = k n ) 
for all co 

n k 

|E[JV B (w)|Z ,...,^]-E[JV n (w)|Z ,...X]l< E "{Z'^Z'-), (8) 

j=i+i 

where Z'- Z'- if C4*„ (Zj) = C4*>„ (Z'-). 

a) Description of Coupling: Suppose we have {^} i=1 and \Z\ }\ =l - To obtain Z'- +l and Z'- +1 , 
starting from states Z'- and Z- we run copies Ojj}- >;L an d {Xji\ •>! °f coupled chains individually faithful 
to p T q . Then Z- is the state corresponding to the first time {Zj,Y^} hits a context in G. Similarly 
for Zj . Specifically, the chains {Y^} and {Yj' i }. >1 are coupled as follows. We generate a number Uj\ 
uniformly distributed G [0, 1]. Given {Z- and Z^ ) with suffixes u and v respectively in G, we let G u G A 
(and G v similarly) be the set of symbols in A defined as in Q. We split the interval from to 1 as 
follows: for all a £ A, we assign intervals r(a) of length min {q(a\u), q(a|v)}, in the following order: we 
first stack the above intervals corresponding to a £ G u n G v (in any order) starting from 0, and then we 
put in the intervals corresponding to all other symbols. Now let, 

(Y n ,Y.[) = (a,a) if Uj± £ r(a). 
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g(a 2 |u) = 0.3 



g(oi|u) = 0.7 



g(aa|v) = 0.6 



g(ai|v) =0.4 





u v 



(a) 



u (ai) =0.3 



r(a 2 ) = 0.3 



r(ai) = 0.4 



r v (a 2 ) = 0.3 



r(a 2 ) = 0.3 



r(ai) = 0.4 



1 - C(A) 



C[A) 



(b) 



Fig. 4. (a) The conditional probabilities with which Yji and Yj 1 have to be chosen respectively are g(-|u) and g(-jv). The line on 
the left determines the choice of Yj 1 and the one on the right the choice of Yji. For example, if Uji is chosen uniformly in [0,1], 
the probability of choosing Yji = a± is q(ai\u). Instead of choosing Yji and Yji independently, we will reorganize the intervals 
in the lines so as to encourage Yji = Yji. (b) Reorganizing the interval [0, 1] according to the described construction. Here 
r(ai) = min {q(a\ ju), q(a\ |v)} and similarly for rifli). If Uji falls in the interval corrsponding to r(ai), then (Yji, Yji) = 
(ai, ai). If Uji > C(A) in this example, then (Yji, Yji) = (ai, 02). When Uji is chosen uniformly in [0,1], the probability Yji 
outputs any symbol is the same as in the picture on the left, similarly for Yj 2 . 



Let 

C(A) = J>(a) = ^mm{q(a\Zj),q(a\Zj)}. (9) 

a<=A a£A 

be the part of the interval is already filled up. Thus if Uji < C(A), equivalently with probability C(A), 
the two chains output the same symbol. We use the rest of the interval [C(A),1] in any valid way to 
satisfy the fact that Yj 1 is distributed as p T q (-\Z'-) and Yj 1 is distributed as p T q (-\Z'-). For one standard 
approach, for all a assign 

f\i(a) = (q(a\u) — q(a\ v)) + = max (g(a|u) — q(a\v), 0} 

and similarly r v (a). Note that only one of r u (a) and r v (o) can be strictly positive and that for all a, 

r(a) + r u (a) = q(a\u) while r(a) + r v (a) = q(a\v). Therefore, 

^r u (a) = ^r v (a) = l-C(A). 

aeA aeA 

An example of such construction for binary alphabet is illustrated in Fig. [4] in which we have assumed 
G u n G v = {ai}. We will keep two copies of the interval [C(^4),l], and if Uji > C(A) we output 
(Yji, Yji) based on where Uji falls in both copies. We will stack the first copy of [C(A), 1] with intervals 
of length r u (a) for all a and the second copy of \C(A), 1] with intervals length r v (a) for all a. We say 
Uji € (r u (a),r v (6)) if Uji E r u (a) in the first copy and Uji G r v (b) in the second copy. Furthermore, 

(Yj^Y'-i) = (a,b) if Uji e (r u (o),r v (6)). 

1) If ^(Z'jY-i) e G and c f (Z'-Y- x ) G G, we have Z' j+1 and Z'- +v 
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2) If Y'- x = Yj[ but only one of Cf(Z' j Yj 1 ) G G and c^(Z'.'Y^) e G, then we have one of Z' j+1 and 
Z- +1 . To get the other, we continue (according to transitions defined by p T q ) only its corresponding 
chain till we get a good state. 

3) If Y'- x = Yj V c^Z'-Y'^) £ G and c^Z-Y'^) ^ G, we need to continue both chains. We generate 
Yj 2 , Y- 2 as we did for the first samples — by generating a new random number Uj2 uniform in [0, 1], 
and by coupling as in ([9]) the distributions q^Z'-Y'^) and q(-\Z"-Y'- A ) respectively. And continue in 
this fashion so long as the samples in the two chains remain equal but do not hit good contexts. This 
will be case that will be most important for us later on. 

4) If Yji 7^ Yjl at any point and neither chain has seen a good state yet, we just run the chains 
independently from that point on for how long it takes each to hit a good aggregated state. 

b) Analysis of coupling: For any r, let Z r ~ Z r denote the following event that is a subset of case 
(1) in the list above, 

jl^ = Y r 'i and CftZ'^xYrx) £ G and f^j-(Z" r _ x Y^ x ) e Gl\. 
From (|7]) and using Lemma [5} we can easily show 

U)(Z' r ~ Z'r\Z' r _ x , Z'^_ x ) >V e 0-~ VkJ, (10) 

where the 1 — z/^ term comes because the parameter r] s is defined on the aggregated parameters, but Y'- 
and Yj evolve according to p T q . Furthermore, if Z\ ~ Z'l for the k n consecutive samples j — k n + 1 < 

i < j, then we have Z- « Z- . To proceed, once Z- « Z- , we would like them to coalesce tighter in every 

, k n +l ,, , k n ,, 

subsequent step, namely we want for all 1 < I < n, Z- +l m Z- +l . Starting from Z- « Z- , one way we 

/ k 4-1 n , ir I n 

can have Z-,, "ra Z-,-, is if Z-,, ~ Z' ,,, or if the chains \Y'- \.^_ and fYl v }.^, evolve through a 

sequence of m > 1 steps before hitting a context in G on the m'th step with Y- x = Y- { for each I < to. 
This is the situation in case 3 of the list above, but in addition in each step I, 

c Akn {{z y 3l }\ =l ) = CAkn ({Zj, y;^ =1 ), 

i k a 

since Z ■ « Z ■ . Therefore, both chains will hit a common good context in G in to steps. But in addition 
we will also have 

/ k„+rn it 

Z j+i ~ Z i+i- 

Because of the way we have set up our coupling, the probability 

^(Yji = Y ji\ z 'j ^ z j) = X] min [<l( a \ z 'j)^Q(, a \ z 'j)} 

> J]g(a|c t (Z;.))(l-^J 

= l ~ u k n , 

where q and q are the model parameters associated with p T q and p T respectively. Similarly 



w ' Y j(i+i) ~ Y j(i+i) 



z '^Zl{Y^ = {Y^ i= ^>l-v kn+l . 



Therefore (no matter what to is), 

, k„ + l „, , k n 

l=k„ 



u(z- ^ z]\z' 3 _ x k z'-_ x ) > f[ (i - Vl ) > i - $ A 
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Note that we use a very similar argument to obtain for all I 

oo 

lo(3£' < £ s.t. Z' j+e K ^Z] +v \Z'j k Z'j) > JJ(l_^)>l- 

l=k n 

because the above event, {3£' < £ s.t. Z- +£ , « ^j+i'} can happen by going through a sequence tighter 
and tighter coalesced transitions of p T q (no matter in how many steps we saw contexts in G). And we 
can easily strengthen the above to say for all I, 



co(Z 



.i+> 



k„+£ il . i k„ ;/. 

- z J+ ,\z^z 3 )>i-^ kn 



(11) 



for the same reason. Indeed, we can further strengthen the above statement to note that after we see 
Z- « Z^ for any /, the chance of ever diverging is 

u(3l > s.t. Z' j+l ji Z"^ X \Z\ I Z-) < $ t . (12) 
c) Bound on Martingale differences: Let £ n be as defined in the statement of this Theorem. In an 

ill I l n II 

abuse of notation, we say the chains \Z^\ and \Z i } have merged if for any j, Z- « Z- , and let r be the 

smallest number such that Z T « Z T . The probability r > t£ n can be upper bounded by observing splitting 
the first t£ n samples in blocks of length £ n , and observing that the probability the chains merge in any 
single block is, using ([10]) k n times and then ([IT} > 77*" (1 - 3>fc„)(l - Vk n ) hn > »7* n (l - ^fcj/ 4 - Thus, 

u(r>U n ) < (l-77^(l-$ fe J/4y. 

Furthermore, Er is less than the expected number of blocks before the chains merge in any single block, 
thus, 



Er < 



7£»(l-*fcJ' 



Furthermore, note that for all j > i + 1, 



56 Z"j) = LoiZ'j 96 and r < j) + w(Z_J 96 and r > j) 

(<*) , tn „ 1 

< $e + u(Za 96 Z, and r > j) < - + uj(t > j). 
j j n 

where inequality (a) above follows because Z T s5 Z T by definition and from §V2\ . Finally we upper 
bound ^ 



£ ^Z'^Z-)<n. l -+ "(T>j)<l + VT<l + - j — 



44 



j"=i+l i=i+l g K ™ J 

The final step of the proof comes by bounding the value of Vo = E[iV n (w)|Zo] by a coupling argument 
very similar to the one above. Suppose {Z n } and {Z n } are coupled copies of {Z n }, where {Z n } starts 
from state Zq, while {Z'^} starts from a state chosen randomly according to the stationary distribution of 
{Zn}. The same analysis holds, and from Property [2] 



KG) 



< 1 + 



Mr, 



r/Ml-<h 



as well. (The theorem only has at most constant confidence if the upper bound above > t/2yn.) The 
Theorem now follows by a direct application of Azuma's inequality. □ 
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Remark Note that if the dependencies die down exponentially, namely /(£) = 7* for some < 7 < 1, 



We have shown how to use data generated by potentially slow mixing Markov sources (or channels) 
to identify those states for which naive approaches will estimate both parameters and functions related 
to stationary probabilities. To do so, we require that the underlying Markov source (or channel) be well 
behaved — in the sense that dependencies cannot be completely arbitrary, but die down eventually. In such 
cases, we show that even while the source may not have mixed (explored the state space properly), certain 
properties can related to contexts w, namely # w or /x(w)//x(C7) be well estimated, if |w| grows as O(logn). 
Surprisingly, we saw that it is quite possible that estimates related to contexts w may be good, even when 
estimates for suffixes of w fail. The reason is that Theorem [6] depends not on the source mixing, but on the 
dependencies dying off. This work also uncovers a lot of open problems. The above results are sufficient to 
say that some estimates are approximately accurate with high confidence. A natural, but perhaps difficult, 
question is whether we can give necessary conditions on how the data must look for any estimate to be 
accurate. Secondly, since we can control the evolution of the channel (or source), how — if possible — do 
we force the channel to explore other states better? Namely, if the channel had feedback, how could we 
adapt the source to better guage channel properties? Practically speaking, how do we best leverage these 
results in operating channels with long memory? 
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